fix(producer,engine): gate drawelement on ancestor background-image + tail verify sample - #2247
Conversation
… tail verify sample
miga-heygen
left a comment
miga-heygen
left a comment
There was a problem hiding this comment.
#2247 — gate drawelement on ancestor background-image + tail verify sample
Verdict: LGTM 🟢 (CI failures look like format/lint — likely needs a rebase or format pass)
Two independent fixes for the same failure mode (30.9 dB body-gradient damage):
1. Ancestor background-image detection. detectAncestorBackgroundImage walks the ancestor chain of the composition root (body → html → wrapper divs) checking inline styles AND <style> rule selectors via querySelectorAll through linkedom. Correctly ignores elements INSIDE the root (those are painted by drawElement). The regex BACKGROUND_IMAGE_DECL_PATTERN matches gradient/url but not plain background-color. Test coverage is thorough: rule-selected body gradient, url() on html, inline style, class-selected wrapper, negative cases (child elements, plain color, no root).
When detected, compileStage.ts gates drawElement off — same shape as the existing 3D-transform and mix-blend-mode gates. Env override HF_FAST_CAPTURE_ANCESTOR_BG=true for R&D bypass.
2. Tail-pinned verify sample grid. computeDeVerifySampleFractions replaces the old (i+1)/(k+1) grid with first k-1 evenly spaced + last pinned at 95%. The old grid's final sample sat at 80%, missing late-onset damage (a body-gradient drop starting at ~79% passed verification). Tests prove: strictly increasing, within (0,1), correct single-sample case, empty on disabled.
Both fixes attack the same measured failure from different directions — detection prevents the bad path, tail sampling catches it if it slips through.
Review by Miga
miguel-heygen
left a comment
miguel-heygen
left a comment
There was a problem hiding this comment.
Reviewed current head 0809c98 and diff. Ancestor background-image detection walks only the root ancestor chain and tests cover body/html/inline/class wrappers; sample grid is bounded and tail-pinned at 95%. All completed checks are green, but the required Windows test job is still pending, so I am not approving prematurely. Please re-request review once that required job is green.
james-russo-rames-d-jusso
left a comment
james-russo-rames-d-jusso
left a comment
There was a problem hiding this comment.
Reviewed at 0809c98.
Both fixes land at the right layer.
Fix 1 — ancestor background-image gate. The rootcausing is exactly right: drawElementService's ancestor fill walks up for the nearest non-transparent backgroundColor, so a gradient/url on <body>/<html>/a wrapper reads as transparent and a deeper solid paints instead. Detection walks the ancestor chain for both inline styles AND <style> rules (resolved via querySelectorAll — so class-selected wrappers are covered, not just literal html/body). Backgrounds inside the root are deliberately not matched. Same gate shape as the 3D/mix-blend neighbors (packages/producer/src/services/render/stages/compileStage.ts:265-278), HF_FAST_CAPTURE_ANCESTOR_BG=true bypass for R&D. The gate flows through as deCompileGate = "ancestor_background_image" to perfSummary.drawElement.compileGate, so route-decline aggregates are already differentiable — good.
Fix 2 — tail verify sample. computeDeVerifySampleFractions(k) moves the last sample to 95%. The k=4 default becomes [0.25, 0.5, 0.75, 0.95] — retains three equidistant heads (very close to the old grid's density there) and pins a real tail. The k=1 → [0.95] and k<=0 → [] edges are covered in the test set. The strictly increasing within (0, 1) invariant test is nice.
BACKGROUND_IMAGE_DECL_PATTERN matches gradient( and url(. -webkit-image-set(), cross-fade(), element(), paint() — the CSS Images-4 image-notation menagerie — are not matched. All rare in HF authoring by convention, so noting it not flagging it.
LGTM from my side.
jrusso1020
left a comment
jrusso1020
left a comment
There was a problem hiding this comment.
Approving per the channel stamp-flow: Rames-D cleared it and the required Windows check is now green at 0809c98 (that pending check was the only thing holding it).
… tail verify sample (heygen-com#2247) ## What Two fixes from adversarial testing of the DE parallel router (10 hostile comps, routed vs screenshot-baseline PSNR). The router itself held — both bugs are in general drawElement fast capture, and one slipped past self-verify. ### 1. Compile gate: ancestor background-image (`producer`) `drawElementService`'s per-frame ancestor fill replicates what lies behind the captured subtree by walking up the DOM for the nearest non-transparent **`backgroundColor`**. A background-**image** (`linear-gradient`, `url()`) on `body`/`html`/a wrapper reads as transparent in that scan, so a deeper ancestor's solid color paints instead wherever the subtree leaves pixels uncovered. Measured repro: body `linear-gradient` + html solid color + an element shrinking late in the comp → DE paints the html purple instead of the body gradient. 30.9 dB min frame vs baseline, visually unmistakable. Identical damage single-worker and parallel — general DE bug, in every wild DE render matching this (very common) authoring pattern. Fix: `detectAncestorBackgroundImage()` in the compiler (DOM-aware — inline styles on the root's ancestor chain + `<style>` rules resolved via `querySelectorAll`, so class-selected wrappers are covered; backgrounds *inside* the root are deliberately not matched). New compile gate `ancestor_background_image`, same shape as the 3D/mix-blend gates, bypass `HF_FAST_CAPTURE_ANCESTOR_BG=true`. ### 2. Self-verify tail sample (`engine`) The verify grid sampled at `(i+1)/(k+1)` → [20/40/60/80]% of the timeline. The damage above starts at ~79% and peaks after the last sample — verification **passed** on output that bottomed at 30.9 dB (threshold 32 dB would have caught it, it just never looked there). Fix: `computeDeVerifySampleFractions()` — first k−1 samples evenly spaced, last pinned at 95%. Default k=4 grid becomes [25/50/75/95]%. Kills the whole late-onset damage class, not just this repro. ## Validation - Repro comp (body gradient + shrink reveal): now gates → baseline route, 54.7 dB avg vs ground truth (was 41.6 avg / 30.9 min with the purple surround) - Control comp (nested stacked fades, routed): still routes, verify grid `[90, 180, 270, 342] of 360`, passes, 60.6 dB avg — unchanged - Full adversarial matrix context: 6/10 comps routed clean (49–68 dB min), blend/3D gated correctly, animated-canvas damage caught by verify at 16.5 dB with clean revert, video comps route legitimately (frames pre-extracted) - Tests: 7 new detection cases (`htmlCompiler.test.ts`), 5 new grid cases (`frameCapture-verifySampleFractions.test.ts`); `compileStage.test.ts` + `frameCapture.test.ts` suites green 🤖 Generated with [Claude Code](https://claude.com/claude-code)
5 participants
What
Two fixes from adversarial testing of the DE parallel router (10 hostile comps, routed vs screenshot-baseline PSNR). The router itself held — both bugs are in general drawElement fast capture, and one slipped past self-verify.
1. Compile gate: ancestor background-image (
producer)drawElementService's per-frame ancestor fill replicates what lies behind the captured subtree by walking up the DOM for the nearest non-transparentbackgroundColor. A background-image (linear-gradient,url()) onbody/html/a wrapper reads as transparent in that scan, so a deeper ancestor's solid color paints instead wherever the subtree leaves pixels uncovered.Measured repro: body
linear-gradient+ html solid color + an element shrinking late in the comp → DE paints the html purple instead of the body gradient. 30.9 dB min frame vs baseline, visually unmistakable. Identical damage single-worker and parallel — general DE bug, in every wild DE render matching this (very common) authoring pattern.Fix:
detectAncestorBackgroundImage()in the compiler (DOM-aware — inline styles on the root's ancestor chain +<style>rules resolved viaquerySelectorAll, so class-selected wrappers are covered; backgrounds inside the root are deliberately not matched). New compile gateancestor_background_image, same shape as the 3D/mix-blend gates, bypassHF_FAST_CAPTURE_ANCESTOR_BG=true.2. Self-verify tail sample (
engine)The verify grid sampled at
(i+1)/(k+1)→ [20/40/60/80]% of the timeline. The damage above starts at ~79% and peaks after the last sample — verification passed on output that bottomed at 30.9 dB (threshold 32 dB would have caught it, it just never looked there).Fix:
computeDeVerifySampleFractions()— first k−1 samples evenly spaced, last pinned at 95%. Default k=4 grid becomes [25/50/75/95]%. Kills the whole late-onset damage class, not just this repro.Validation
[90, 180, 270, 342] of 360, passes, 60.6 dB avg — unchangedhtmlCompiler.test.ts), 5 new grid cases (frameCapture-verifySampleFractions.test.ts);compileStage.test.ts+frameCapture.test.tssuites green🤖 Generated with Claude Code